Variational Inference for Latent Variable Modelling of Correlation Structure

Latent variable models have played an important part in unsupervised learning, where the goal is to capture the structure of some complicated observed data in a set of variables that are somehow simpler. PCA or Factor Analysis, for example, models high dimensional data using lower dimensional and uncorrelated latent variables. The value of the latent variable represents some underlying unobserved explanation of the observation. Often, the latent variables can be interpreted as having some kind of meaning, which is useful for exploratory data analysis. Many models follow the same principle of explaining the value of an observation by mapping from a “simple” latent space to complicated observations. Much work has been done to increase the flexibility of these mappings. The GPLVM (Lawrence and Hyvrinen, 2005), for example, uses a non-linear function with a Gaussian process prior to map from the latent space.